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Abstract 

Image super-resolution is a process to enhance image resolution. It is widely used in medical 
imaging, satellite imaging, target recognition, etc. In this paper, we conduct continuous mod¬ 
eling and assume that the unknown image intensity function is defined on a continuous domain 
and belongs to a space with a redundant basis. We propose a new iterative model for single im¬ 
age super-resolution based on an observation: an image is consisted of smooth components and 
non-smooth components, and we use two classes of approximated Heaviside functions (AHFs) to 
represent them respectively. Due to sparsity of the non-smooth components, a Li model is em¬ 
ployed. In addition, we apply the proposed iterative model to image patches to reduce computation 
and storage. Comparisons with some existing competitive methods show the effectiveness of the 
proposed method. 
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1 Introduction 

Image super-resolution (SR) is to estimate a high-resolution (HR) image from one or multiple low- 
resolution (LR) images. When there is only one low-resolution input, it is called single image super¬ 
resolution. The other case is called multiple image super-resolution. Compared to multiple image 
super-resolution, single image super-resolution is more practical when only one low-resolution image 
is available. Obviously, it is more challenging. In this paper, we address single image super-resolution. 
We can only collect low-resolution images sometimes because of the limitation of hardware devices 
and high cost. For instance, it needs more cost and time to get high-resolution images in medical 
imaging MRI. Due to long distance and air turbulence, we can not get high-resolution images for 
synthetic aperture radar (SAR) and satellite imaging. In target recognition, we may not always get 
high-resolution videos to make recognition accurately. Thus, it is important and useful to develop a 
more effective image super-resolution algorithm. 

There are mainly four categories of image super-resolution methods: interpolation-based methods, 
learning-based methods, statistics-based methods and other methods. These methods are however 
not completely independent. For instance, some of statistics-based methods may involve learning 
strategies. 

Nearest-neighbor interpolation and bicubic interpolation are two classical interpolation methods. 
Nearest-neighbor interpolation estimates intensity of an unknown location using that of its nearest 
neighbor point. It however often creates jaggy effect. Bicubic interpolation is to interpolate unknown 
intensity by utilizing a cubic kernel, and may cause blur effect. More interpolation methods have 
been proposed recently [SI [361 Ea H EH EH ES]. In [21], it introduces a contour stencils method to 
estimate the image contours based on total variation (TV) along curves. This method can distinguish 
lines of different orientations, curves, corners, etc., with a computationally efficient formula so that the 
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resulted high-resolution images preserve image details well. In |46j . Wang et al. propose a fast image 
super-resolution method on the displacement field. This method is based on two stages, one stage uses 
an interpolation technique to pick up the low-frequency image and the other employs a reconstruction 
method to recover local high-frequency structures. 

Recently, many image super-resolution methods are learning-based [inmiiMiiiiiMiiiHiisiisii 
[soiisaisnjiiiiiniiniiisiisijisniiiiisiiiiiiiEiiMiisiiisiiiiiii]- Learning-based methods need 
two large training data sets, one formed by low-resolution images and the other by the corresponding 
high-resolution images. We learn a relation between the two training data sets, and then apply the 
relation to a given low-resolution image to obtain a high-resolution image. They rely heavily on 
the selection of training data and involves expensive computation. They are not single image super¬ 
resolution approaches in strict sense as they require extra training data. In |50] . Yang et al. propose 
a sparse signal representation method for single image super-resolution. This method jointly trains 
two dictionaries for the low-resolution and high-resolution image patches, then applies the generated 
dictionaries to a low-resolution input to get the corresponding high-resolution output. In |12| . Dong 
et al. first apply a deep learning method for single image super-resolution. The method learns a 
mapping between low-resolution and high-resolution images. The mapping is represented by a deep 
convolutional neural network (CNN) that takes the low-resolution image as the input and outputs the 
high-resolution image. 

Statistics-based methods such as Maximum a Posterior (MAP) and Maximum Likelihood estimator 
(MLE) are a popular tool for image super-resolution [HI El ES] . Fattal [15] utilizes statistical edge 
dependency relating edge features in low and high resolution images to preserve sharp edges. In [14] . 
an alternative approach using Li norm minimization and a bilateral prior based robust regularization, 
is proposed by Farsin et al. 

Many other methods have also been proposed for image super-resolution. Examples are hybrid 
method m, pixel classification method mm, frequency technique [4] , reconstruction method [2711281 
Esmniiii] and others [SSlEllIilEzllH]. 

Most existing image super-resolution methods are based on discrete models. In this paper, we pro¬ 
pose a continuous model based on approximated Heaviside functions (AHFs) for single image super¬ 
resolution. Because an image normally consists of smooth components and non-smooth components, 
we model an image as a sum of two classes of AHFs, one representing smooth components and the 
other depicting non-smooth components, e.g., step edges. Due to the sparsity of non-smooth compo¬ 
nents, we design a Li regularization model and solve it by alternating direction method of multipliers 
(ADMM). An iterative model based on the Li model is proposed to preserve more details. We apply 
the coefficients, computed by the iterative model, to generate high-resolution images at any upscaling 
factors. In particular, the iterative model is applied to image patches, aiming to get cheap computa¬ 
tion and storage. For images with smooth backgrounds, the proposed method may cause ring artifacts 
and we develop a strategy utilizing image gradient to reduce them. Numerical experiments show the 
proposed method is a competitive method comparing with existing excellent methods. 

To the best of our knowledge, this is the first work to use AHFs on image super-resolution. The 
proposed method can preserve not only edges, but also the high-frequency details on non-edge regions. 
The proposed method is completely single image super-resolution method without using training data. 

The organization of this paper is as follows. In Section 2, we review Heaviside function and show 
why we can use it for image super-resolution. In Section 3, we present the proposed method, including 
the corresponding new model, new algorithm and other strategies. Visual and quantitative experiments 
are given in Section 4. Finally, we draw conclusions in Section 5. 


2 Heaviside function 


Heaviside function, or Heaviside step function, is defined as follows (see Fig. EKa)) 


(l){x) 


0, X < 0, 

1, X > 0, 


( 1 ) 


which is singular at x = 0. In practice, we usually use its approximation, called approximated Heaviside 
function (AHF), such as and ^ -I- iarctan(|) which approximate to (j){x) when ^ —7> 0. In this 
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(a) 


(b) 


Figure 1: (a) Heaviside function; (b) Two approximated Heaviside functions with ^ = 0.7 (blue solid 
line) and ( = 0.05 (black dash line), the smaller ^ the sharper edge (color images are better visualized 
in the pdf file). 


paper, we use the following approximated Heaviside function, 

, . 1 1 .X. 

ip(x) = - H—arctan( —), 

2 TT q 


( 2 ) 


where ^ G TZ actually controls the smoothness. The smaller the sharper edge (see Fig. [Hb)). 

From |29j . we know that any function in i3p([0, l]'^), p G [1, oo), has a best approximation by linear 
combinations of m characteristic functions of half-space, and m is any positive integer. Let Hd be a 
set of functions on [0,1]'^ defined as 

i?d = {/: [0,l]‘'^7^:/(x)=V'(v•x-bc),ve7^^ce7^}, (3) 


where tp is an approximated Heaviside function. Hd is the set of characteristic function of closed 
half-spaces of 

Theorem 1 (see [2^) For any positive integer d, define spaUmHa as {J2iLi'-ai'4’ivi-x+Ci)}, where 
LOi GTZ and Vi G and Ci G TZ, then it is known that Um^j^+spaUmFld is dense in (£p([0,1]'^), || • ||p), 
P e [l,oo) . 

Theorem 2 (see [52]) For every positive integers m,d and every p G [1,cxd), spaUmHa is approxi¬ 
mately a compact subset of (£p([0,1]'^), || • ||p). 

In practical computing, one only can afford to use spaUmHd for a finite m. 

In our work, we focus on image super-resolution, i.e., d = 2. We assume the underlying image 
intensity function / is defined on [0,1]^ and / G £p([0,1]^) with p G [1, oo), / can be approximated by 
the following equation, 

m 

( 4 ) 

where coj G TZ, Vj G TZ^ and Cj G TZ. We discreatize Vj = {{cos9t,sin9t)', t = 1, 2, • • • , fc} to denote k 
different directions, and cj = {^, ' ) 1} to denote discrete positions, m = kq, z = {x, y)'. We hx 

q as the total number of pixels of the input image. Furthermore, {'0(vj • z -|- Cj)}fTi with a specific 
^ is called a class of AHFs. In particular, we can describe edges of different orientations 9t at the 
locations Cj when setting a small For an image L G we assume it is a discretization of 

intensity function / on [0,1]^, i.e., = f{xi,yj),Xi = ^,yj = ^, i = 1,2, ■ ■ ■ ,ni, j = 1,2,--- ,n 2 . 

We can rewrite Eq. (|4|) as matrix-vector form, L ra vj'w, where G G 77", w G 7Z"^ with 

n = nin 2 ,rn = kq. Once we have computed coefficient w, we can make image super-resolution with 
upscaling factor s possible by the equation where 'L G 77^^™ is obtained similarly with 'k, 
N = s^nin2. 
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Figure 2: Left panel: surface images of if) under ^ = 0.1 and nine random parameter pairs (0,c); 
Right panel: the corresponding 2D intensity images. From left to right and then from top to bottom: 

( 47 r 51 \ ( 477 - 25 *' ( 47 t- 175 \ ( &2L 135 'i ( Stt ]_\ /gw 25/Sw _5_'\ / 8 w 75 \ / Sw 75 \ /ff,^ hpttpr 
1 5 ’ IO 24 F t 5 ’ 64/’ i 5 > 2560 1 5 > 10240 i 5 ’ 20 I 5 ) 32 F V 5 > 640 t 5 ) 256 0 t 5 > 128 '' 1 ^ ueiiei 

visualization, we make slightly rotation to some surface images). 



Figure 3: Left panel: surface images of ip under ^ = 10“^ and the same nine parameter pairs (0, c) 
as in Fig. [2 Right panel: the corresponding 2D intensity images, (for better visualization, we make 
slightly rotation to some surface images). 


4 








































3 The proposed model and its solution 

3.1 The proposed Li model and its solution 

From Fig. [5] and Fig. |31 it is easy to know that the corresponding AHFs with big ^ represent smooth 
components well, and the corresponding AHFs with small ^ depict non-smooth components such as 
edges well. An image is generally consisted of smooth components and non-smooth components, thus 
we represent an image with two classes of AHFs, one with big parameter to represent smooth 
components (forming 'I'l) and the other with small ^2 to depict non-smooth components (forming ^' 2 ). 
By this strategy, the vector-form image L, still denoted by I, can be approximated by the following 
discrete formula, 

L Ri tPl/Sl -I-4'2/32) (5) 

where 'll, 1/2 G and /3i, ^2 € are representation coefficients. The following task is to 

establish a model and compute the representation coefficients /3i and {32 , and then apply the computed 
coefficients to get super-resolution images hy f — 'll/?! -I- '12/32, where 'll, 'I 2 G are generated 

similarly as '1 using finer grids. 

For an image, smooth components can be mainly represented by 'li/3i, non-smooth components 
such as edges are depicted by '12/32- Since non-smooth components are sparse in images. For instance, 
edges are not everywhere but distribute sparsely in most images. Thus we use h sparsity on ^ 2 - The 
optimization model is 

\\L — 'li/3i — 'I 2 / 32 II 2 + ^i||/3i||2 + ^2||^2||i, (6) 

P1.P2 

where Ai, A 2 are regularization parameters, L represents a low-resolution input. 

Since Li term is not differentiable, we make a variable substitution for ^ 2 , and rewrite model ([5|) 
as 

rnin ||T — d>i^i —'I 2 / 32 II 2 + Ai||/3i||2 + A 2 ||m||i, s.t., u = 132, (7) 

Pl,P 2 

where u is the substitution variable. In particular, Eq. 0 can be rewritten as 

mm||L-^'^|l 2 -f Ai||A^||2-1A2||m|1i, s.t., u = B[3, (8) 

where '1 = ('li,'l 2 ), (3 = (/3i,/32)', A = (1,0) and B = (0,1). The optimization problem dH) is 
separable w.r.t (3 and u. There are many methods to solve (8). We select alternating direction method 
of multipliers (ADMM)[^ HJl [^. 

The augmented Lagrangian of Eq. (jS]) is 

£(/3,u, h) = \\L - '1/3||2 + Ai||A/ 3||2 + A2||w||i + |||u - + b\\l, (9) 

where b is Lagrangian multiplier with proper size. The problem of minimizing C(j3, u, b) is solved by 
iteratively and alternatively solving the following two subproblems: 

/3-subproblem : min \\L — 'L/ 3||2 + Ai||A/ 3||2 + ^\\u — B/3 -I- 6 II 2 , (10) 

M-subproblem : minA 2 ||m||i + — BP + b\\l. (11) 

u 2 

We get the following ADMM algorithm for problem ([8]): 


5 


Algorithm 1 


Input: Given low-resolution image L, 'I'l, '^ 2 ) Ai, A 2 , p 

Output: /3 

1. /3W ^ 0, ^ 0, ^ 0 

2 . while not converged do 

3. fc •(— fc -I- 1 

4. <r- solve subproblem (flUl) for u = b = 

5. ■<— solve subproblem (ITT]) for /3 = b = 

6. ^ -h (uW 

7. End while. 

The /3-subproblem m can be solved by least squares method: 

P = K-\ (12) 

where /3 e K = -f e r = f (u -h 6) -h e 7 ^ 2 '"xl. 

The it-subproblem dD has a closed form solution for each m (see [47] ') 

A 2 

Ui = shrink((Bf3)i — bi, —), (13) 

P 

where shrink{a, b) = sign(a)uiax{\a — b\, 0) and 0.(0/0) = 0 is assumed. 

3.2 The iterative AHF method based on the proposed model 

Even though model (jb)) takes different behaviors of smooth components and non-smooth components 
into consideration, we still see some blur in the high-resolution image. We find that the difference 
image L — DH^^\ where is the resulted high-resolution output by model ([6]) and Zl is a down- 
sampling operator, contains some residual edges. To pick up more edges and details and thus to 
make results less blurry, we consider the difference L — as a new low-resolution input L of 

model ([6|) to recompute a residual high-resolution image This process is repeated until the resid¬ 
ual edges are small enough. The sum of H^^'> and its residual high-resolution images is the resulted 
super-resolution image. This iterative strategy can recover more image details (see Fig. m). The fol¬ 
lowing Algorithm 2 is the proposed iterative AHF algorithm for single image super-resolution. We use 
bicubic downsampling D in our experiments although Algorithm 2 can work for general downsampling. 

Algorithm 2 (Single image super-resolution via iterative AHF method) 

Input: one low-resolution image (vector-form): L G , Ai > 0, A 2 > 0, s: upscaling factor. 

t: maximum number of iteration. 

Output: high-resolution image (vector-form): H G TT.'^xi 

1. Construct matrices 'I '2 G 77,"^™ on coarse grids and 'f'l, 'I '2 G 77^^™ on fine grids (see Section 3.1), 
where N = s^n. 

2. Initialization: = L. 

for k = 1: T 

a. Compute the coefficients: ) = argmin||L('') -^iPi -'I'2/32||i + Ai||/3i|||-I-A 2 II/ 32 II 1 . 

b. Update the high-resolution image: where ■ 

c. Downsampling H^^'> to coarse grid: L = DE[^^'>. 

d. Compute residual: — L. 

end 

3. Assemble the high-resolution outputs: S = ^ . 

4. Compute the final high-resolution image: 

H = S + conv{E,p), 

where conv represents a convolution operator, and p is a Gaussian kernel with small size. 
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Figure 4: Illustration of step 2 of Algorithm 2 using “wing” image; (a) the sum image of = 1, 2, 3 

(we set 3 iterations here) ; (b) the computed image for the first iteration. For better visualization, we 
add 0.3 to the intensities of and to brighten them. From the last two images, we can see 
that and pick up image details. 



(a) H (b) S (c) conv{E,p) 


Figure 5: Illustration of step 4 of Algorithm 2 on pathes of a “wing image; (a) the sum image S + 
conv{E,p) (see step 4 of Algorithm 2); (b) the image S; (c) the image conv(E,p) which contains some 
edges and high-frequency information away from edges. For better visualization, we add 0.3 to the 
intensities of conv{E,p) to brighten it. 


Note that step 2.a in Algorithm 2 is solved by Algorithm 1. From Fig. [SJ we know that E represents 
some edges. In particular, we use a Gaussian kernel p with size 5x5 and standard derivation 1 to make 
a convolution to reduce the oversharp information on the non-edge regions. In addition, although we 
introduce some parameters in the algorithm, they are not sensitive and easy to select. We will give 
a remark on parameter selection in Section 4. This is the proposed iterative method for single image 
super-resolution. 

3.3 Apply the proposed method to image patches 

We have proposed the iterative AHF method for single image super-resolution, but it can be very 
expensive if we apply it to a whole image. For instance, if one low-resolution image has size 100 x 100 
and the upscaling factor is 2, then 4'i, 'i >2 with 10 angles should have size 10, 000 x 100, 000. It is very 
expensive to implement Algorithm 2 with big non-sparse matrices. In our work, we apply the iterative 
AHF method to image patches, so that we can reduce computation and storage significantly. We set 
patch size to be 6 x 6 and overlap to be 2 in our work. Moreover, pixels on the boundary will be 
handled by bicubic interpolation. 

In summary, we use a set of Heaviside functions with sharp edges to represent sharp image edges 
which can be viewed as a kind of image high-frequency information. In particular, applying the 
proposed method to each image patches may make image high-frequency information more significant. 
This is the reason why the proposed method can pick up image high-frequency information. 
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4 Numerical experiments 

In this section, two kinds of test images are utilized to illustrate the performance of different methods. 
One is low-resolution images without high-resolution ground-truth. The other is low-resolution images 
downsampled from known high-resolution images, for which we can make quantitative comparisons. 
All experiments are implemented in MATLAB(R2010a) on a laptop of 3.25Gb RAM and Intel(R) 
Core(TM) i3-2370M CPU: @2.40 GHz. 

We compare our method with some competitive image super-resolution methods: nearest-neighbor 
interpolation, bicubic interpolation, iterative back-projection method (denoted as “BP” [H]); a edge- 
directed interpolation (denoted as “01’TIP” [34]), an edge-guided interpolation (denoted as “06’TIP” 
[55]'). a kernel regression method (denoted as “07’TIP” HD), a fast upsamling method (denoted as 
“OS’TOG” [3D)> TV-based super-resolution method (denoted as “ll’JMIV” [5]), three state-of-the- 
art learning-based methods (denoted as “04’CVPR” 0, “lO’TIP” [5D| and “12’BMVC” [3]) and three 
state-of-the-art interpolation methods (denoted as “ll’IPOL” HI], “ll’SIAM” HD and “14’TIP” jUj). 

For gray images, we can apply the proposed Algorithm 2 to them directly. For color images such 
as RGB, we transform test images in RGB color space to “Ycbcr” color space which is very popular in 
image/video processing. “Y” represents luma component, “Cb” and “Cr” are blur-difference and red- 
difference components, respectively. We only apply the proposed algorithm to the illuminance channel 
because humans are more sensitive to luminance changes. In addition, we interpolate the color layers 
(Cb, Cr) using bicubic interpolation. After getting the upsampled images in “Ycbcr” color space, we 
transform them back to the original RGB color space for visualization. 

We use root-mean-square error (RMSE) on illuminance channel to evaluate numerical performance, 


RMSE = 




N 


N 


'^{hi - 


(14) 


i=l 


where h, h are the vector-form of ground-truth image and resulted super-resolution image, respectively. 
Note that although RMSE is a common numerical criterion, smaller RMSE can not always promise 
better visual results. 

Remark on parameter selection: For the parameters in Algorithm 1 and Algorithm 2, we set 
p = 10“"^, Ai = 10“^, A 2 = 10“®, T = 3. Matrices with ^ = 10“^ and 4>2,'I'2 with ^ = 10“"^. 

We set 12 angles evenly distributed on [0,27r], i.e., 9 — {0, f ^,7r, ^}. We 

fix q as the total number of pixels of the low-resolution patch, i.e., q = 36 for 6x6 patches. Note that, 
fine tuning of parameters for some images may get better results. However, we unify the parameter 
selection to illustrate the stability of the proposed method. 


4.1 Super-resolution for generic images 

From Fig. [5] we test low-resolution images without known high-resolution images. We compare 
the proposed method with nearest-neighbor interpolation, bicubic interpolation, two state-of-the-art 
learning-based methods (“lO’TIP” [50] and “12’BMVC” [3]), a fast upsampling method (“OS’TOG” 
[ST] ! and three state-of-the-art interpolation-based methods (“ll’IPOL” [22], “11’SIAM” HD and 
“14’TIP” [46]). The upscaling factors are all 3. From the resulted images, super-resolution images by 
bicubic interpolation generate blur effect. “OS’TOG” performs well on image edges, but it smooths 
the details on the non-edge regions. The two learning-based methods (“lO’TIP” and “12’BMVC”) 
perform comparable visually. In addition, “ll’IPOL” and “11’SIAM” also show similar performance. 
In particular, “14’TIP” shows better property for image edges, but it smooths the image details on the 
non-edge regions. The proposed method outperforms the other methods, especially for image details 
on the non-edge regions. 

In Fig. [7] and Fig. [8] low-resolution test images are generated by downsampling known high- 
resolution via bicubic interpolation. Because ground-truth images are known, we can evaluate quanti¬ 
tative results using RMSE. The upscaling factors are set to be 4 for “baboon” and “forest”, 3 for “field” 
and 2 for “leopard”. From these figures, we know that the results of “OS’TOG” generate sharp edges 
but smooth out details on non-edge regions. TV-based method “ll’JMIV” [S] also obtains sharp edges 





ll’SIAM 12’BMVC 14’TIP Ours 




Bicubic 


08'TOG 


llIPOL 


11’SIAM 12'BMVC 14’TIP Ours 


Figure 6: Results of “butterflywing” (first row) and “landscape” (second row), upscaling factors are all 
3. From left to right: low-resolution images, results of bicubic interpolation, “lO’TIP” [50], “08’TOG” 
[37], “fl’IPOL” [22], “ll’SIAM” [2T], “12’BMVC” [3], “14’TIP” [^ and the proposed method (color 
images are better visualized in the pdf file). 


but has oil-painting effect. Results of iterative back-projection “BP” [23; the learning-based method 
(neighborhood embedding “04’CVPR” |9]) and the kernel regression method “07’TIP” [41] show sig¬ 
nificant blur. Because the proposed method can be viewed as an interpolation method, we compare 
our method with three state-of-the-art interpolation methods (“ll’IPOL” [22], “11’SIAM” [21] and 
“14’TIP” [46]). In addition, we also compare the proposed method with two modern learning-based 
methods (“lO’TIP” [S^ and “12’BMVC” [3])- From Fig. [7] and Fig. [S] the proposed method obtains 
smaller RMSE than the three state-of-the-art interpolation methods and the two learning-based meth¬ 
ods, and generates high-resolution images with more image details. Actually, learning-based methods 
need extra training data that can be viewed as extra prior information while our method does not 
require any extra data. Note that the interpolation-based method “14’TIP” [46] keeps very sharp 
image edges and runs quite fast, but it obtains bad RMSE and smooths image details significantly on 
the non-edge regions (see the close-ups). Furthermore, more quantitative results can be found in Tab. 
[T] It also demonstrates the effectiveness of the proposed method. 
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LR Bioibic (19.69) 08'TOG (19.54) IIMMIV (19.81) ll'IPOL (19.32) 



ll’SIAM (19.28) 12’BMVC (19.28) 14’TIP (21.51) Ours (19.20) 




Figure 7: Visual results and the corresponding RMSE of “baboon” and “forest” by upscaling factors of 
4. Compared methods: bicubic interpolation, “BP” [55], “04’CVPR” |5|, “08’TOG” “ll’JMIV” 
|S], “ll’IPOL” HD, “ll’SIAM” [5T], “12’BMVC” |3], “14’TIP” gS], and the proposed method. 


Table 1: RMSE of more test examples 


Example (factor) 

Bicubic 

07’TIP |4T] 

08’TOG [37] 

lO’TIP [50] 

Ours 

lena(2) 

5.10 

9.91 

6.98 

4.09 

3.84 

landscape(2) 

10.75 

13.55 

11.74 

10.09 

9.63 

purplebutterfly (2) 

13.03 

16.85 

14.19 

11.69 

11.43 

leaf(3) 

17.51 

18.23 

17.55 

17.39 

17.28 

clog(3) 

2.12 

3.76 

1.95 

1.93 

1.85 

fruits(3) 

4.13 

9.09 

3.55 

3.75 

3.53 

babyface(3) 

5.12 

8.44 

4.86 

4.87 

4.74 

train(4) 

12.98 

15.85 

12.64 

12.85 

12.62 
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LR Ground Nearest (13.82) Bicubic (12.56) 



ll’SIAM (12.16) 12'BMVC (12.17) 14 TIP (13.86) Ours (12.08) 




Ground 08'TOG (16.44) 07'TIP (20.25) ll'IPOL (14.65) 


ll’SIAM (14.40) 12’BMVC (14.28) 14‘TIP (18.46) Ours (14.23) 


Figure 8: Visual results of “field” (upscaling factor 3) and “leopard” (upscaling factor 2) and the corre¬ 
sponding RMSE. Compared methods: nearest-neighbor interpolation, bicubic interpolation, “07’TIP” 
[11], “08’TOG” [37], “lO’TIP” [50], “IFIPOL” [22], “IPSIAM” [2T], “12’BMVC” [3], “14’TIP” [46], 
and the proposed method. 
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(a) LR 


(b) using original E 


(c) using updated E„ 


Figure 9: Results of “butterfly” by an upscaling factor of 2. Image “butterfly” is an image with the 
smooth background; (a) low-resolution image; (b) super-resolution image by the proposed method 
using original E; (c) super-resolution image by the proposed method using updated Enew 


4.2 Super-resolution for the images with smooth backgrounds 

The proposed method performs well for natural images. It is a better method than the compared 
methods. However, the proposed method encounters a drawback: ring artifacts along the large scale 
edges of images which have smooth backgrounds (see Fig. [91(b)). The images with smooth backgrounds 
are generally obtained from single lens reflex (SLR) camera or other specific scenes. The ring artifacts 
mainly come from the added non-smooth components (see step 4 of Algorithm 2). In this section, 
we utilize a method without ring artifacts to make a mask, aiming to discard the ring artifacts of 
non-smooth components E. We learn that bicubic interpolation will not cause ring artifacts (see the 
resulted images in section 4.1) and run very fast. Thus we use bicubic interpolation as our intermediate 
method to make the mask. Actually, we only need to update the E in the step 4 of Algorithm 2 by 
Enew which is obtained from the following equations, then we can reduce the ring artifacts significantly. 

Enew = Mask. * E, (15) 


where 


Mask = 


0 , 

1 , 


if 0 < Gij < t, 
otherwise. 


(16) 


where Gij is the vector norm of gradient at the location {i,j) of image B; B = Bicubic{L,s) where 
‘‘‘‘Bicubic” represents implementing bicubic interpolation to a low-resolution image and L is a low- 
resolution image, s is an upscaling factor; notation stands for dot product; t is a thresholding 
value, and we set t — 0.05 in our work. Obviously, this strategy reduces the ring artifacts of super¬ 
resolution images (see Fig. EKc)). 

In Fig. [TOl we apply the proposed method to images “comic” and “face” which have smooth 
backgrounds. The upscaling factors are set to be 2 and 4, respectively. We can find more details from 
the close-ups in Fig. (TOl From the figure, the proposed method reduces ring artifacts significantly and 
performs best, both visually and quantitatively. 

In Fig. 111! we compare the proposed method with some competitive methods. The upscaling factor 
is 2. The test image “butterfly” has obvious smooth background. The results of bicubic interpolation 
and other two interpolation methods “Ol’TIP”, “06’TIP” show significant blur effect. Although the 
result by “08’TOG” and “14’TIP” keeps sharp edges well, they make image details of non-edge regions 
oversmoothing. The learning-based method “lO’TIP” obtains the smallest RMSE and the competitive 
visual result. The proposed method preserves image details of non-edge regions better and overcomes 
ring artifacts significantly. In addition, RMSE of our method is almost the smallest one. 


Parameters selection: The proposed method involves many parameters, e.g., Ai, A 2 , ^ 1 , ^ 2 , patch 
size, etc. However, they are easy to select because the results are not sensitive to the selection of pa¬ 
rameters (see ^ 1 , ^2 in Tab. [Hand Ai, A 2 in Tab. [3]). Actually, choosing suitable parameters is always 
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Ground 08’TOG (6.50) ll’JMIV (6.68) Ours (6.36) 


Figure 10: Results of “comic” (first row, upscaling factor 2) and “face” (second row, upscaling factor 
4) and the corresponding RMSE. First row: ground-truth, bicubic interpolation, the learning-based 
method “lO’TIP” [50] and the proposed method; Second row: ground-truth, the fast upsampling 
method “08’TOG” [37]) TV-based method “ll’JMIV” [5] and the proposed. Our method gets smallest 
RMSE and better visual results. 


a difficulty to many algorithms. Tuning empirically is a popular way for determining parameters. In 
our work, we obtain the parameters empirically. Eor instance, for Ai and A 2 , we first fix one parameter 
Ai, and then tune A 2 with 10 times change. For instance, we fix Ai = 10“^, then tune A 2 by 10“^, 
10“"^, 10“^, 10“®, 10“^, etc. When we find A 2 = 10“® is the best one for Ai, then tune A 2 in the rang 
of [10“®,10“^], e.g., 5 X 10“® and 5 x 10“^. Actually, the results around A = 10“^ and A 2 = 10“® 
also do not show obvious difference (see Tab. El). Fine tuning of parameters might lead to slightly 
better results. But to make it simple, we set Ai = 10“^, A 2 = 10“® in all test examples. Under this 
setting, the results are already good enough for comparisons. Similarly, we select the other parameters 
according to this way. 


Table 2: RMSE around = 0.1 and ^2 = 10 ^ 


^2 

0.8 X 10-4 

1 X 10-4 

1.2 X 10-4 

0.8 X 10-4 

12.09 

12.08 

12.07 

1 X 10-4 

12.10 

12.08 

12.07 

1.2 X 10-4 

12.09 

12.08 

12.08 


Computation time: From Fig. fT^ a). we know that the computation time does not increase signifi¬ 
cantly when upscaling factor is increased from 2 to 9. Using our non-optimized Matlab code, it is below 
10 seconds for a low-resolution image 60 x 60. From Fig. UlTb), when the size of low-resolution image 
is from 30 x 30 to 110 x 110, the computation time is about increased from 3 seconds to 31 seconds. 
We can conclude that the computation time mainly depends on the size of low-resolution image but 
not so significantly on the scaling factor, since the main computation is to compute coefficients P via 
the low-resolution image. Note that, there is a lot of room to speed up the speed. We can use cmex 
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Groluicl 


Bicubic (6.36) 


OI TIP (10.24) 



06 TIP (9.9.5) 


08 TOG (8.08) 


lO'TIP (5.14) 


ll'IPOL (5.32) 


14'TIP (9.47) Ours (5.15) 


Figure 11: Results of “butterfly” by an upscaling factor of 2. First row: ground-truth, results of 
bicubic interpolation, an edge-directed interpolation “01’TIP” [34]; Second row: results of an edge- 
guided interpolation “06’TIP” |^, a fast upsampling method “08’TOG” [37] ; the learning-based 
method “lO’TIP” [50]; Third row: results of two interpolation methods “ll’IPOL” [22] and “14’TIP” 
[46] . and the proposed method. 


Table 3: RMSE around Ai = 10 ^ and A 2 = 10 ® 


Ai 

A2 

0.8 X 10-2 

1 X 10-2 

1.2 X 10-2 

0.8 X 10 "“ 

12.09 

12.08 

12.08 

1 X 10 -“ 

12.09 

12.08 

12.08 

5 X lO "'’' 

12.10 

12.09 

12.08 
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(a) (b) 


Figure 12: (a) Computation time vs. upscaling factor for low-resolution image with size 60 x 60; (b) 
Computation time vs. size of low-resolution image, the size of low-resolution image is increased from 
30 X 40 to 110 X 110 and the upscaling factor is always set to be 5. The experimental computer is with 
3.47CB RAM, Inter(R) Core(TM) i3-2130 CPU, @3.40GHz. 


in Matlab to speed up the code that involves a lot of loops. We can also use parallel computing as the 
computation is done patch by patch. 

The relation between model (6) and model (6) combined with iterative idea: It is necessary 
to illustrate the relation between model (6) and model(6) combine with our iterative idea. Actually, 
there is only fine difference, especially in image details and edges, between model (6) and model (6) 
combined with iterative idea. Due to small magnitude of the difference, there is no significant difference 
between the two methods. In Fig. [T2]we show an example. From Fig. [T31 it is easy to know that 
the proposed model (6) combined with the iterative AHF method performs similarly RMSE with the 
proposed model (6). In addition, the error maps between the two strategies are almost same. 

5 Conclusions 

In this paper, we casted image super-resolution problem as an intensity function estimation problem. 
We assumed the underlying intensity function, defined on a continuous domain and belonging to a 
space with redundant basis, could be approximately represented by two classes of approximated Heav¬ 
iside functions (AHFs). The representation coefficients were computed by the proposed iterative AHF 
method using only one low-resolution image. We then applied the coefficients to get high-resolution 
images. To reduce computation and storage, we applied the proposed iterative AHF method to image 
patches. For images with smooth backgrounds, we designed a strategy utilizing image gradient to 
reduce ring artifacts along large scale edges. The method can be applied to any upscaling factors and 
needs no extra data for learning. In particular, we also discussed the parameter selection and com¬ 
putation time. Many experiments show that the proposed approach outperforms existing competitive 
methods, both visually and quantitatively. 
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RMSE = 19.20 RMSE = 19.28 



RAISE = 17.81 RAISE = 17.92 


Figure 13: First column: results of model (6) with iterative AHF method; Second column: results of 
model (6) without iterative AHF method; Third column: error maps (for better vision, we add extra 
intensity 30/255 to true error maps). Upscaling factors: 4. 
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